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Rising employer demand for skilled workers has driven 
efforts to better align occupational training programs to 
industry needs.! Yet, even as the demand for skilled workers 
increases, less than half of students who enter occupational 
training programs receive a credential within six years.” 
Community-based organizations are partnering with 
community colleges and occupational training providers to 
find faster and more effective ways to train those in need of 
basic skills instruction in math, reading, or job skills, and 
provide wraparound supports to help students succeed in 
college or occupational training programs. 


Project QUEST (Quality Employment through Skills Training) 
provides comprehensive support services to help 
participants complete occupational training programs 

at local community colleges and professional training 
institutes, pass certification exams, and obtain well-paying 
jobs in targeted sectors of the local economy. Project QUEST 
started in 1992 in San Antonio, Texas, and has been adapted 


in other locations in Texas and Arizona under different 
names, including VIDA (Valley Initiative for Development and 
Advancement) in the Rio Grande Valley of Texas, Capital 
IDEA in Austin, Texas and Houston, Texas, Project ARRIBA 
(Advanced Retraining & Redevelopment Initiative in Border 
Areas) in El Paso, Texas, and JobPath in Tucson, Arizona.? 


This What Works Clearinghouse™ (WWC) report, part of the 
WWC’s Postsecondary Career and Technical Education topic 
area, explores the effects of Project QUEST on education 
and labor market outcomes. The WWC identified five 
studies of Project QUEST. Three of these studies meet WWC 
standards.‘ The evidence presented in this report is from 
studies of the impact of Project QUEST on students enrolled 
in occupational training programs—including Asian, Black, 
White, and Hispanic students—in community colleges and 
professional training institutes in urban, suburban, and 
rural settings. 


What Happens When Students Participate in Project QUEST?® 


The evidence indicates that implementing Project QUEST: 


e¢ is likely to increase industry-recognized credential, 
certificate, or license completion 


¢ may increase credit accumulation 


e may result in little to no change in short-term 
employment, short-term earnings, medium-term 
employment, medium-term earnings, and 
long-term earnings 


e may decrease postsecondary degree attainment 


Findings on Project QUEST from three studies that meet 
WWC standards are shown in Table 1. The table reports 
an effectiveness rating, the improvement index, and the 
number of studies and students that contributed to the 
findings. The improvement index is a measure of the 
intervention’s effect on an outcome. It can be interpreted 
as the expected change in percentile rank for an average 
comparison group student if that student had received the 
intervention. 


Table 1. Summary of findings on Project QUEST from studies that meet WWC standards 


Study findings Evidence meeting WWC standards (version 4.0) 


Improvement index 


Outcome domain Effectiveness rating (percentile points) Number of studies Number of students 
certs, orteense compleion __—_~estve effects +45 2 1301 
Credit accumulation Potentially positive effects +14 1 958 
Medium-term employment No discernible effects +8 1 343 
Medium-term earnings No discernible effects +7 1 410 
Long-term earnings No discernible effects +4 1 410 
Short-term earnings No discernible effects +4 1 410 


Study findings Evidence meeting WWC standards (version 4.0) 


Improvement index 


Outcome domain Effectiveness rating (percentile points) Number of studies Number of students 
Short-term employment No discernible effects -1 2 1,555 
Postsecondary degree attainment Potentially negative effects -3 2 1,301 


Note: The improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the 
intervention. For example, an improvement index of +15 means that the expected percentile rank of the average comparison group student would increase by 15 points if the 
student received Project QUEST. The improvement index values are generated by averaging findings from the outcome analyses that meet WWC standards, as reported by 
Elliott & Roder (2017), Rolston et al. (2017), and Juniper et al. (2020). A positive or negative improvement index does not necessarily mean the estimated effect is statistically 
significant. The industry-recognized credential, certificate, or license completion outcomes reported in these studies include earned a vocational certificate or license within 6 
years after random assignment, receipt of a college credential: any credential within 24 months of random assignment, receipt of a college credential: any certificate within 24 
months of random assignment, and receipt of a credential from a college within 24 months of random assignment, another education or training institution by time of survey, 
or a licensing/certification body by time of survey. The credit accumulation outcomes reported in these studies include total college credits earned within 24 months of random 
assignment. The medium-term employment outcomes reported in these studies include employment (year-round) year 5 after random assignment. The medium-term earnings 
outcomes reported in these studies include earnings (average annual earnings) year 5 after random assignment. The long-term earnings outcomes reported in these studies 
include earnings (average annual earnings) year 7 after random assignment. The short-term earnings outcomes reported in these studies include earnings (average annual 
earnings) year 3 after random assignment. The short-term employment outcomes reported in these studies include employment 4 quarters after completion and employment 
(year-round) year 3 after random assignment. The postsecondary degree attainment outcomes reported in these studies include earned a college degree within 6 years of 
random assignment and receipt of a college credential: any degree within 24 months of random assignment. The effects of Project QUEST are not known for other outcomes 
within the Postsecondary Career and Technical Education topic area protocol, including technical skill proficiency and long-term employment. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the Postsecondary Career and Technical Education topic area protocol 
(version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship between 
effectiveness ratings and the statements used in this report: 


Effectiveness rating Rating interpretation Description of the evidence 


Positive (or negative) effects The intervention is likely to change an Strong evidence of a positive (or negative) 
outcome effect, with no overriding contrary evidence 


Potentially positive (or negative) effects The intervention may change an outcome Evidence of a positive (or negative) effect with 
no overriding contrary evidence 


No discernible effects The intervention may result in little to no No affirmative evidence of effects 
change in an outcome 


Mixed effects The intervention has inconsistent effects Evidence includes studies in at least two of 
onan outcome these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


How is Project QUEST Implemented? 
The following section provides details of how Project QUEST 


is implemented. This information can help educators iden- Comparison group: In the three studies that contrib- 
tify the requirements for implementing Project QUEST and ute to this intervention report, students in the compar- 
determine whether implementing this intervention would ison group could access any of the services available 
be feasible at their institutions. Variations in the implemen- in their local communities such as enrolling in occupa- 
tation of Project QUEST, VIDA, and Capital IDEA are noted tional training programs on their own or seeking out 
below. Information presented in this section comes from job placement assistance at the college or American 


the three studies that meet WWC standards (Elliott & Roder, Job Center. 
2017; Juniper et al., 2020; Rolston et al., 2017), the Project 
QUEST, VIDA, and Capital IDEA websites, and correspon- 
dence with the developer. 


skills, postsecondary credentials, and/or certificates 

needed to obtain well-paying jobs and meet the needs of 

¢ Goal: Project QUEST aims to provide comprehensive employers in targeted sectors of the local economy. 
support services to help local adult residents gain the 


N 


e Target population: Project QUEST targets adults 18 years 
or older who hold a high school diploma or equivalent 
and demonstrate at a minimum an 8" grade level in 
reading and 6" grade level in math on the Tests of Adult 
Basic Education (TABE). Participants must also be 
available to enroll full-time in an occupational training 
program.® Project QUEST recruits individuals who are 
interested in but are not currently enrolled in college. 
VIDA targets participants who demonstrate at a minimum 
a 10" grade level in reading, writing, and math, many of 
whom are already enrolled in college but need additional 
supports to complete their college coursework. All three 
interventions target their efforts on recruiting individuals 
who are unemployed, underemployed, meet federal 


attend weekly group meetings led by a counselor that 
focus on life skills, time management, study skills, test- 
taking techniques, critical thinking, conflict resolution, 
and workforce readiness skills. Participants who need 

to improve their basic reading and math skills can 
complete basic skills coursework prior to enrolling in the 
occupational program. 


Frequency and duration of service: Participants 
typically complete their occupational program within one 
to three years, depending on the length of the program. 


Intervention components: Project QUEST is comprised 
of several components, including a comprehensive 
intake process, full-time enrollment in an occupational 


poverty guidelines, or are on public assistance. program, basic skills instruction, intrusive advising, job 
placement support, financial support, and partnerships 
with occupational training partners and employers. Refer 


to Table 2 for additional details. 


¢ Method of delivery: Project QUEST is a community-based 
organization that partners with colleges, professional 
training institutes, and employers. Participants enroll 
full-time in an occupational training program. They 


Table 2. Components of Project QUEST 


Key component Description 


Comprehensive intake Project QUEST conducts a comprehensive intake process to determine participant eligibility, baseline skills in reading and 
process math, and interests. The intake process, which is similarly rigorous for VIDA and Capital IDEA, includes six steps. 

1. Attend Orientation: Interested individuals attend an orientation that provides an overview of the program and program 
expectations. 

2. Complete Application, Placement Tests, and Essay: Individuals complete an application detailing their education 
and employment history, family situation, and childcare arrangements; they take placement tests; and they write a career 
exploration essay. These three components are used by counselors to assess the applicant's fit for the program. 

3. Complete Career Aptitude and Assessment Test, College Placement Test, and Interview: Applicants take a career 
aptitude and assessment test to help evaluate whether their skills and interests match their career goals. Project QUEST 
uses the System for Assessment and Group Evaluation (SAGE) and VIDA uses the Harrington-O’Shea Career Decision- 
Making System. Applicants also take the ACCUPLACER college placement test to identify any remediation needs. Finally, 
applicants meet with a counselor to discuss and apply for eligible funding sources. 

4. Develop Academic and Financial Plans: Applicants meet with an assigned counselor to create an academic plan to 
map out the required courses and timeline for completion, develop a budget, and address potential barriers to 
program completion. 

5. Submit Required Documentation: Applicants submit required documentation such as proof of income and family size to 
verify their eligibility for funding sources. 

6. Participate in Final Interview: Applicants attend an interview with a senior staff person who reviews their academic and 
financial plans and determines whether the applicant is accepted into the program. 


Full-time enrollment 
in an occupational 
training program 


Participants are required to enroll full-time in an occupational program. The occupational programs take one to three years to 
complete, depending on the program. Project QUEST requires participants to be enrolled in a certificate program at an area 
community college or professional training institute in manufacturing and trades, healthcare, or information technology. VIDA 
requires participants to be enrolled in an associate's or bachelor’s degree program at an area college in nursing, allied health, 
technology, manufacturing, business, education and social services, or specialized trades. Capital IDEA requires participants 
to be enrolled in a degree or certificate program at an area community college in information technology, healthcare, or other 
leading industry such as professional trades or applied technologies. 


Basic skills instruction Project QUEST offers basic skills instruction—or QUEST Prep classes—for participants who need to improve their basic 
reading and math skills prior to enrolling full-time in an occupational program. QUEST Prep classes are offered 25 hours 
per week at no cost to participants. Participants can complete the basic skills coursework part-time. VIDA and Capital IDEA 
offers a College Prep Academy for participants who are unlikely to pass the Texas State Initiative Assessment (TSIA) college 
entrance exam. Participants attend the College Prep Academy full-time at no cost to participants for 16 weeks (VIDA) or 12 
weeks (Capital IDEA). 


Intrusive advising Counselors or career navigators hold mandatory weekly group meetings on college campuses focused on topics designed 

to support the achievement of academic and employment goals such as life skills, time management, study skills, test-taking 
techniques, critical thinking, conflict resolution, and workforce readiness skills. The weekly sessions typically last 45 minutes 
to an hour and allow counselors to check in with students on their academic progress. Additionally, counselors provide 
individual advising for academic and non-academic issues, motivation and emotional support, and referrals to other social 
services for assistance with utility bills, childcare, food, and other supports. Project QUEST counselors are available for 
individual advising as needed. VIDA requires participants to attend individual advising once a month for up to an hour. Capital 
IDEA career navigators meet individually with participants at the start of each semester. 


Key component 


Description 


Job placement support 


Project QUEST’s employment coordinators and Capital IDEA career navigators provide additional job placement assistance 
including resume and cover letter writing, job search and application assistance, and interview preparation. Employment 
coordinators and career navigators also inform participants about upcoming job fairs and job openings. VIDA does not offer 
extensive employment services and job placement upon program completion. Rather, counselors refer participants to career 


services centers on their college campus or the local American Job Center for assistance. 


Financial support 


Participants receive financial support for academic and non-academic expenses that can be barriers to learning. Project 


QUEST offers financial support to cover tuition and fees—100 percent for the first year and 50 percent for subsequent 
years. Proof of class attendance is required to receive tuition assistance; counselors verify class attendance weekly by 
auditing participant attendance sheets signed by occupational instructors. Project QUEST also offers financial support for 
books, uniforms, licensing fees, and other education-related expenses. Additionally, Project QUEST offers some funding for 
supportive services such as transportation, medical care, eye exams, utilities, and childcare. 


VIDA provides up to $5,500 of financial support in a 12-month period to cover tuition and fees, along with financial support 
for books, transportation, licensing fees, uniforms, other education-related expenses including childcare related to attending 
school or unexpected emergencies such as car repairs. Routine assessments of financial needs are conducted at intake and 


at the start of each semester. 


Capital IDEA covers the cost of tuition, fees, books, uniforms, and other materials required by the occupational training 
program. Additionally, Capital IDEA covers eye exams and eyeglasses, and offers participants access to free public 
transportation. Emergency financial assistance is available for utility bills, mortgage, rent assistance, and childcare. 
Participants are encouraged to work part-time while completing the occupational training program. 


Partnerships Project QUEST and Capital IDEA partner with area community colleges and professional training institutes which offer 
associate’s degrees, certificates, and licenses in occupational programs. Employment coordinators build and maintain 
industry connections with employers. VIDA does not partner with employers directly. 

Other VIDA requires participants to complete two community service activities per semester. There are no community service 


requirements for Project QUEST or Capital IDEA participants. 


What Does Project QUEST Cost? 


This preliminary list of costs is not designed to be 
exhaustive; rather, it provides educators an overview of 
the major resources needed to implement Project QUEST. 
The program costs described in Table 3 are based on the 
information available as of May 2021. 


As reported in Roder & Elliott (2018), the average cost of 
Project QUEST was $10,501 per participant over an average 
of 22 months. Project QUEST staff accounted for 41 percent 
of the costs; financial support for students accounted 

for 23 percent; tuition accounted for 22 percent; and 
administrative and fundraising costs accounted for 13 
percent. The operating costs of Project QUEST are financed 
by a combination of private and public funding. The primary 
funder is the City of San Antonio, Texas. Other funders 


Table 3. Cost ingredients for Project QUEST 


include Bexar County, Texas and the Charles Stewart Mott, 
Meadows, and Annie E. Casey Foundations. 


Rolston et al. (2017) reported that the cost of VIDA was 
approximately $13,750 per participant over an average of 
two and a half years. VIDA received funding from local 
economic development corporations and cities and counties 
in the Rio Grande Valley region of Texas. VIDA received 
funds directly from the Kresge, Laura and John Arnold, 
Meadows, and Hearst Foundations, and an Open Society 
Foundations grant to Abt Associates to double the number 
of participants included in the evaluation. 


Juniper et al. (2020) did not break down the per participant 
cost of Capital IDEA. 


Cost ingredients Description Source of funding 
Staff to provide direct services to students, including counselors, basic skills instructors, and other ; 

sea support staff such as employment coordinators for Project QUEST. Project QUEST 
Facilities costs include space for administrative staff and for conducting intake, support services, and Project QUEST; 

Facilities basic skills instruction. Area colleges and professional training institutes provide space for the delivery —_ colleges; professional 
of occupational coursework and for conducting weekly group meetings. training institutes 

Equipment and Standard information technology (IT) equipment such as computers, phones, printers, and photocopiers Project QUEST 

materials for staff is required. 

Other Other costs include financial support for students to cover academic and non-academic expenses that Project QUEST 


can be barriers to learning, program fundraising, and administrative costs. 


For More Information: 
About Project QUEST 


Project QUEST, Inc. 

800 Quintana Road, Building 8 

San Antonio, Texas 78211 

Attn: Hugo Hernandez, Manager of Service and Delivery 


Email: hugo@questsa.org Web: https://questsa.org/ Phone: 210-630-4690 


About the cost of the intervention 


Roder, A., & Elliott, M. (2018). Escalating gains: The elements of Project QUEST’s success. Economic Mobility Corporation. 
https://economicmobilitycorp.org/wp-content/uploads/2018/06/Elements-of-Project-QUESTs-Success. pdf 


Rolston, H., Copson, E., & Gardiner, K. (2017). Valley Initiative for Development and Advancement: Implementation and early impact 
report, OPRE Report #2017-83. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and 


Families, U.S. Department of Health and Human Services. 


https://www.acf.hhs.gov/sites/default/files/documents/opre/vida_implementation_and_early impact_report_final_b508.pdf 


Research Summary 


The WWC identified five studies that investigated the 
effectiveness of Project QUEST (Figure 1): 


¢ 2 studies meet WWC group design standards without 
reservations 


¢ 1study meets WWC group design standards with 
reservations 


¢ 2 studies are ineligible for review 


The WWC reviews findings on the intervention’s effects 

on eligible outcome domains from studies that meet 
standards, either with or without reservations. Based on this 
review, the WWC generates an effectiveness rating, which 
summarizes how the intervention impacts, or changes, a 
particular outcome domain. The WWC reports additional 
supplemental findings, such as total college credits earned 
at additional time points, on the WWC website 
(https://whatworks.ed.gov). 


These supplemental findings and findings from studies that 
either do not meet WWC standards or are ineligible for 
review do not contribute to the effectiveness ratings. 


The three studies of Project QUEST that meet WWC group 
design standards reported findings on credit accumulation; 
industry-recognized credential, certificate or license 
completion; postsecondary degree attainment; short-term 
earnings; short-term employment; medium-term earnings; 
medium-term employment; and long-term earnings. No 
other findings in the studies meet WWC group design 
standards within any outcome domain included in the 
Postsecondary Career and Technical Education topic area.” 
Citations for the three studies reviewed for this report are 
listed in the References section, which begins on page 15. 
Citations for the two studies that are ineligible for review 
and the reasons the WWC determined they were ineligible 
are also listed in the References section. 


Figure 1. Effectiveness ratings for Project QUEST 


2 studies meet WWC study meets WWC 0 studies do not 2 studies are 


standards without standards with 
reservations reservations 


meet WWC TasVike lle) (-miele 
standards review 


Contribute to effectiveness ratings Do not contribute to effectiveness ratings 


Project QUEST has positive effects on industry-recognized credential, certificate, or license completion 


The WWC determined that two studies that meet WWC group design standards without reservations (Elliott & Roder, 2017; 
Rolston et al., 2017) showed evidence of a positive and statistically significant effect of Project QUEST on industry- 
recognized credential, certificate, or license completion. 


Project QUEST has potentially positive effects on credit accumulation 


The WWC determined that one study that meets WWC group design standards without reservations (Rolston et al., 2017) 
showed evidence of a positive and statistically significant effect of Project QUEST on credit accumulation. 


Project QUEST has no discernible effects on medium-term employment 


The WWC determined that one study that meets WWC group design standards without reservations (Elliott & Roder, 2017) 
showed evidence of an indeterminate effect of Project QUEST on medium-term employment. 


Project QUEST has no discernible effects on medium-term earnings 


The WWC determined that one study that meets WWC group design standards without reservations (Elliott & Roder, 2017) 
showed evidence of an indeterminate effect of Project QUEST on medium-term earnings. 


Project QUEST has no discernible effects on long-term earnings 


The WWC determined that one study that meets WWC group design standards without reservations (Elliott & Roder, 2017) 
showed evidence of an indeterminate effect of Project QUEST on long-term earnings. 


Project QUEST has no discernible effects on short-term earnings 


The WWC determined that one study that meets WWC group design standards without reservations (Elliott & Roder, 2017) 
showed evidence of an indeterminate effect of Project QUEST on short-term earnings. 


Project QUEST has no discernible effects on short-term employment 


The WWC determined that two studies that meet WWC group design standards without reservations (Elliott & Roder, 2017; 
Juniper et al., 2020) showed evidence of an indeterminate effect of Project QUEST on short-term employment. 


Project QUEST has potentially negative effects on postsecondary degree attainment 


The WWC determined that one study that meets WWC group design standards without reservations (Elliott & Roder, 
2017) showed an effect size less than -0.25 of Project QUEST on postsecondary degree attainment but this finding was 
not statistically significant, and one study that meets WWC group design standards without reservations (Rolston et al., 
2017) showed evidence of an indeterminate effect of Project QUEST on postsecondary degree attainment. 


Main Findings 


Table 4 shows the findings from the three studies of Project 
QUEST that meet WWC standards. The table includes 

WWC calculations of the mean difference, effect size, and 
performance of the intervention group relative to the 
comparison group. Based on findings from two studies that 
meet WWC standards, the effectiveness rating for industry- 
recognized credential, certificate, or license completion is 
positive effects, indicating strong evidence of a positive effect, 
with no overriding contrary evidence. These findings are 
based on 1,301 students. Based on findings from one study 
that meets WWC standards, the effectiveness rating for 
credit accumulation is potentially positive effects, indicating 
evidence of a positive effect, with no overriding contrary 
evidence. These findings are based on 958 students. Based 


on findings from one study that meets WWC standards, 

the effectiveness rating for medium-term employment, 
medium-term earnings, long-term earnings, and short-term 
earnings is no discernible effects, indicating no affirmative 
evidence of effects. These findings are based on 343 to 

410 students. Based on findings from two studies that 

meet WWC standards, the effectiveness rating for short- 
term employment is no discernible effects, indicating no 
affirmative evidence of effects. These findings are based on 
1,555 students. Based on findings from two studies that meet 
WWC standards, the effectiveness rating for postsecondary 
degree attainment is potentially negative effects, indicating 
evidence of a negative effect, with no overriding contrary 
evidence. These findings are based on 1,301 students. 


Table 4. Findings by outcome domain from studies of Project QUEST that meet WWC standards 


Mean 
(standard deviation) WWC calculations 
Sample Intervention Comparison Mean Effect Improvement 
Measure (study) Study sample size group group difference _—_ size index p-value 
Earned a vocational Full sample 343 72.60 49.10 23.50 0.61 +23 01 
certificate or license (%) 
(Elliott & Roder, 2017) 
Receipt of a college Full sample 958 55.44 45.00 10.44 0.25 +10 <.01 
credential (%): Any 
credential (Rolston et al., 
2017)? 
Receipt of a college Full sample 958 29.10 23.13 5.97 0.19 +7 04 
credential (%): Any 
certificate (Rolston et al., 
2017)° 
Receipt of a credential Full sample 958 64.54 60.72 3.82 0.10 +4 22 
from (%): A college 
within 24 months, 
another education or 
training institution by 
time of survey, ora 
licensing/certification 
body by time of survey 
(Rolston et al., 2017)° 
Outcome average for industry-recognized credential, certificate, or license completion 0.40 +15 eee 
Total college credits Full sample 958 34.04 27.49 6.55 0.35 +14 < 01 
earned within 24 months (17.93) (19.33) 
of randomization 
: ; Statistically 
b 
Outcome average for credit accumulation (Rolston et al., 2017) 0.35 +14 significant 
Employment (Year- Full sample 343 66.60 59.20 740 0.19 +8 0.16 
Round) (%), year 5 after 
random assignment 
Not 
Outcome average for medium-term employment (Rolston et al., 2017)° 0.19 +8 statistically 
significant 
Earnings ($), year 5 after Full sample 410 24,713 20,792 3,921 0.18 +7 0.10 
random assignment (21,346) (21,236) 


Mean 


(standard deviation) WWC calculations 
Sample Intervention Comparison Mean Effect Improvement 
Measure (study) Study sample size group group difference _—_ size index p-value 
Not 
Outcome average for medium-term earnings (Elliott & Roder, 2017)? 0.18 +7 Statistically 
significant 
Earnings ($), year 7 after Full sample 410 27,155 25,228 2,527 0.11 +4 0.28 
random assignment (24,099) (23,347) 
Not 
Outcome average for long-term earnings (Elliott & Roder, 2017) 0.11 +4 Statistically 
significant 
Earnings ($), year 3 after Full sample 410 19,092 17,159 1,933 0.11 +4 0.26 
random assignment (18,072) (16,513) 
Not 
Outcome average for short-term earnings (Elliott & Roder, 2017)? 0.11 +4 statistically 
significant 
Employment (%), Full sample 1,212 75.32 69.00 6.32 0.14 +6 0.07 
4 quarters after 
completion (Juniper et 
al., 2020)° 
Employment (Year- Full sample 343 48.20 55.00 -6.80 -0.17 7 0.21 
Round) (%), year 3 after 
random assignment 
(Elliott & Roder, 2017)? 
Not 
Outcome average for short-term employment -0.01 -1 statistically 
significant 
Earned a college degree Full sample 343 18.20 26.30 -8.10 -0.29 -11 0.10 
(%) (Elliott & Roder, 
2017) 
Receipt of a college Full sample 958 29.71 25.00 4.71 0.14 +6 0.10 
credential (%): Any 
degree (Rolston et al., 
2017)? 
Not 
Outcome average for postsecondary degree attainment -0.07 -3 statistically 
significant 


Notes: For mean difference and effect size values reported in the table, a positive number favors the intervention group and a negative number favors the comparison group. 
The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are given the 
intervention (measured in standard deviations of the outcome measure). An indicator of the effect of the intervention, the improvement index can be interpreted as the expected 
change in percentile rank for an average comparison group student if that student had received the intervention. For example, an improvement index of +4 means that the 
expected percentile rank of the average comparison group student would increase by 4 points if the student received Project QUEST. A positive or negative improvement index 
does not necessarily mean the estimated effect is statistically significant. Some statistics may not sum as expected due to rounding. 

2 Elliott & Roder (2017) did not require corrections for clustering or multiple comparisons nor difference-in-differences adjustments. The standard deviations for the earnings 
outcomes presented here were provided in response to an author query. The non-significant p-values presented here were calculated by the WWC, because categorical 
p-values were only reported in the study for statistically significant differences. This study is characterized as having a statistically significant positive effect on the industry- 
recognized credential, certificate, or license completion outcome domain because the estimated effect is positive and statistically significant. The study is characterized as 
having an indeterminate effect on the medium-term employment, medium-term earnings, long-term earnings, short-term earnings, and short-term employment outcome 
domains because the mean effect size is < 0.25 and not statistically significant. The study is characterized as having a potentially negative effect on postsecondary degree 
attainment because the mean effect size is < -0.25 but not statistically significant. 

> Rolston et al. (2017) required corrections for multiple comparisons in the industry-recognized credential, certificate, or license completion outcome domain. The unadjusted 
means and standard deviations presented here were provided in response to an author query. The p-values and effect sizes presented here were calculated by the WWC. This 
study is characterized as having a statistically significant positive effect on the industry-recognized credential, certificate, or license completion and credit accumulation outcome 
domains because the estimated effect is positive and statistically significant. The study is characterized as having an indeterminate effect on the postsecondary degree 
attainment outcome domain because the mean effect reported is not statistically significant. 


© Juniper et al. (2020) required difference-in-differences adjustments in the short-term employment outcome domain. The p-values and effect sizes presented here were 
calculated by the WWC. The study is characterized as having an indeterminate effect on short-term employment because the mean effect reported is not statistically significant. 
For more information, please refer to the WWC Procedures Handbook, version 4.0, page 22. 


In What Context Was Project QUEST Studied? 


The following section provides information on the setting 
of the three studies of Project QUEST that meet WWC 
standards, and a description of the participants in 

the research. 


WHERE THE STUDIES WERE CONDUCTED 


3 studies, 2,580 students in Texas 


Race 


Fd Ee | 


Black White 3% Asian Other 


Grades 
9 10 11 


Districts: 
Urban 
Suburban 
Rural 


2 © 


Postsecondary (PS) 


This information can help educators understand the context 
in which the studies of Project QUEST were conducted, and 
determine whether the program might be suitable for 

their setting. 


Ethnicity 


Hispanic Non-Hispanic 


Gender 


Female Male 


Details of Each Study that Meets WWC Standards 


This section presents details for the studies of Project QUEST 
that meet WWC standards. These details include the full 
study reference, findings description, findings summary, 

and description of study characteristics. A summary of 
domain findings for each study is presented below, followed 
by a description of the study characteristics. These study- 
level details include contextual information about the study 
setting, methods, sample, intervention group, comparison 
group, outcomes, and implementation details. For additional 
information, readers should refer to the original studies. 


Research details for Elliott & Roder (2017) 


Elliott, M., & Roder, A. (2017). Escalating gains: Project 
QUEST’S sectoral strategy pays off. Economic Mobility 
Corporation. https://economicmobilitycorp.org/wp-content/ 
uploads/2018/01/Escalating-Gains WEB.pdf 


Findings from Elliott & Roder (2017) show evidence of 

a statistically significant positive effect of Project QUEST 
on industry-recognized credential, certificate, or license 
completion (Table 5). This finding is based on an outcome 
analysis that includes 343 students. The findings on 
medium-term employment, medium-term earnings, 
long-term earnings, short-term earnings, short-term 
employment, and postsecondary degree attainment, 
which show evidence of indeterminate effects, are based 
on outcome analysis that includes 343 to 410 students. The 
findings and research details summarized for this study 
come from three related citations, including the primary 
study listed. See the References section, which begins on 
page 15, for a list of all related publications. 


Table 5. Summary of findings from Elliott & Roder (2017) 


Meets WWC Group Design Standards Without Reservations 


Study findings 

Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
Industry-recognized credential, certificate, or license completion 343 0.61 +23 Yes 
Medium-term employment 343 0.19 +8 No 
Medium-term earnings 410 0.18 +7 No 
Long-term earnings 410 0.14 +4 No 
Short-term earnings 410 0.11 +4 No 
Short-term employment 343 -0.16 -7 No 
Postsecondary degree attainment 343 -0.29 -11 No 


Table 6. Description of study characteristics for Elliott & Roder (2017) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Comparison 
group 


Outcomes and 
measurement 


Additional 
implementation 
details 


Meets WWC Group Design Standards Without Reservations. This is a randomized controlled trial (RCT) with low 
attrition. For more information on how the WWC assigns study ratings, please see the WWC Procedures and Standards 
Handbooks (version 4.0) and WWC Standards Briefs, available on the WWC website. 


The study was conducted with residents of San Antonio and Bexar County in Texas. 


Individuals were eligible to participate in Project QUEST if they (a) were 18 years of age or older; (b) had a high school 
diploma or GED®, (c) tested at a minimum at an 8th grade level in reading and 6th grade level in math on the Tests of Adult 
Basic Education (TABE), (d) were interested in but not currently enrolled in college classes, and (e) were able to attend 

one of the healthcare career track programs full-time. Project QUEST targeted recruitment of individuals from lower income 
households-83% of study participants reported household incomes below the poverty level at enrollment. Individuals 
underwent a rigorous intake process which also includes completing the System for Assessment and Group Evaluation 
(SAGE) to assess the match between the individual's career aptitude and interests and their goals and the ACCUPLACER® 
college placement test to identify remediation needs. Individuals who met the program requirements were randomly assigned 
to the intervention or comparison group. 


Of the 410 individuals who participated in the study between April 2006 and October 2008, 207 were randomly assigned to the 
intervention group and 203 to the comparison group. The study used administrative records to report on the short-, medium-, 
and long-term earnings outcomes. For all other outcome measures, the study is based on 343 participants,175 in the 
intervention group and 168 in the comparison group, who completed a baseline survey at the time of enrollment and a follow- 
up survey six years later. Additionally, the study used data from a survey conducted two years after random assignment. 


Of the 410 participants who were eligible to participate in the study and were randomly assigned to either the intervention 
group or comparison group, 14% were Black, 10% were White, and race was not specified for 77% of participants. The 
majority of participants were Hispanic (74%) and female (89%). The average age was 30 years old. The highest degree 
earned for the majority of participants was a GED® (25%) or a high school diploma (70%); 5% had earned a college degree. 
Almost half (45%) had previously attended college but did not earn a degree. The majority (84%) had been employed at 
any time during the past year. The average annual earnings in the past year was $11,722. 


Project QUEST is a workforce development program that serves residents of San Antonio and Bexar County, Texas. For 
the study, program participants were required to enroll full-time in a certificate program at an area community college or 
professional training institute in the field of healthcare such as nursing, medical records coding, or technician programs. 
Project QUEST provides comprehensive support services aimed at helping individuals to (1) complete occupational training 
programs at community colleges and professional training institutes, (2) pass certification exams, and (3) secure relevant 
employment. In addition to attending the occupational training program full-time, participants were required to attend weekly 
group meetings that focused on life skills, including time management, study skills, critical thinking, and conflict resolution. 
Participants received: (a) financial assistance to cover tuition, fees, books, transportation, uniforms, licensing exam 

fees, and tutoring; (b) basic skills instruction to improve their math and reading skills to pass college placement test; (c) 
counseling to address personal and academic concerns and provide motivation and emotional support; (d) referrals to other 
agencies for assistance with utility bills, childcare, food, and other services as needed; and (e) job placement assistance, 
including help with writing resumes and interviewing, and referrals to employers that were hiring. 


Participants in the comparison group could enroll in an occupational training program at an area community college 
or professional training institute and access job placement assistance through the college, available supports in the 
community, or find jobs on their own. However, they did not receive any of the benefits of the Project QUEST program. 


Study authors reported findings on six outcome measures that are eligible for review under the Postsecondary Career and 
Technical Education topic area. One eligible outcome, employment (year-round) year 3 after random assignment, was in 
the short-term employment domain. One eligible outcome, employment (year-round) year 5 after random assignment, was 
in the medium-term employment domain. One eligible outcome, earned a vocational certificate or license within 6 years 
after random assignment, was in the industry-recognized credential, certificate, or license completion domain. One eligible 
outcome, earned a college degree within 6 years after random assignment, was in the postsecondary degree attainment 
domain. One eligible outcome, earnings year 3 after random assignment, was in the short-term earnings domain. One 
eligible outcome, earnings year 5 after random assignment, was in the medium-term earnings domain. One eligible 
outcome, earnings year 7 after random assignment, was in the long-term earnings domain. 


The study also reported findings for employment (year-round) year 4 after random assignment; employment (year-round) 
year 6 after random assignment; earnings year 4 after random assignment; earnings year 6 after random assignment; and 
earnings year 8 after random assignment. Summaries of these findings are available on the WWC website 
(httos:/Awhatworks.ed.gov). The supplemental findings do not factor into the intervention’s rating of effectiveness. 


The operating costs of Project QUEST were financed by a combination of private and public funding. The primary funder 
was the City of San Antonio, Texas. Other funders included Bexar County, Texas and the Charles Stewart Mott, Meadows, 
and Annie E. Casey Foundations. 
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Research details for Rolston et al. (2017) Findings from Rolston et al. (2017) show evidence of a 


Rolston, H., Copson, E., & Gardiner, K. (2017). 

Valley Initiative for Development and Advancement: 
Implementation and early impact report, OPRE Report 
#2017-83. Washington, DC: Office of Planning, Research and 
Evaluation, Administration for Children and Families, U.S. 
Department of Health and Human Services. 
https://www.acf.hhs.gov/sites/default/files/documents/opre/ 


statistically significant positive effect on credit accumulation 
and industry-recognized credential, certificate, or license 
completion (Table 7). These findings are based on an 
outcome analysis that includes 958 students. The finding on 
postsecondary degree attainment, which shows evidence of 
indeterminate effects, is based on an outcome analysis that 
includes 958 students. 


vida implementation and early impact report final b508.pdf 


Table 7. Summary of findings from Rolston et al. (2017) 


Meets WWC Group Design Standards Without Reservations 


Study findings 
Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
Industry-recognized credential, certificate, or license 958 students 0.18 +7 Yes 
completion 
Credit accumulation 958 students 0.35 +14 Yes 
Postsecondary degree attainment 958 students 0.14 +6 No 


Table 8. Description of study characteristics for Rolston et al. (2017) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Meets WWC Group Design Standards Without Reservations. This is a randomized controlled trial (RCT) with low 
attrition. 


The Valley Initiative for Development and Advancement (VIDA) is an adaptation of the Project QUEST model. The study 
was conducted with residents of San Antonio and four counties (Cameron, Hidalgo, Starr, and Willacy) in the Lower Rio 
Grande Valley in Texas. 


Individuals were eligible to participate in VIDA if they (a) were residents of the Rio Grande Valley; (b) were 18 years of age or 
older; (c) had a high school diploma or GED®: (d) were eligible to work in the United States; (e) tested at a minimum at a 10th 
grade level in reading, writing, and math; (f) were unemployed, underemployed, met federal poverty guidelines, or received 
public assistance; and (g) were able to attend an occupational training program at an area community college full-time. Many 
applicants were already enrolled in college but needed additional supports to complete their college coursework. Applicants 
who had not yet enrolled in college were required to take the college entrance exam. Individuals who met the program 
requirements were randomly assigned to the intervention or comparison group. 


Of the 958 individuals who participated in the study between November 2011 and June 2014, 478 were randomly assigned to 
the intervention group and 480 to the comparison group. Data sources include a baseline survey, a follow-up survey starting 
at 15 months after random assignment that occurred on average 20 months after random assignment, and administrative 
records from VIDA and local community colleges that almost all VIDA participants and comparison group members attended. 


Of the 958 participants who were eligible to participate in the study and were randomly assigned to either the intervention 
group or comparison group, 1% were Black, 3% were White, and race was not specified for 96% of participants. The 
majority of participants were Hispanic (96%) and female (71%). Study participants were also older than traditional college 
students. More than 60 percent were age 25 or older, and more than one-fifth were age 35 or older. Virtually all participants 
(99%) had at least a high school diploma or equivalent and more than half (57%) had completed a year or more of college. 
Approximately half (51%) had annual household incomes of less than $15,000, and more than 85% had incomes less than 
$30,000. Most (65%) were not working at the time of random assignment, reflecting their interest in and ability to attend 
school full-time. 
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Intervention 
group 


Comparison 
group 


Outcomes and 
measurement 


Additional 
implementation 
details 


Established in 1995, VIDA is a non-profit, community-based organization, created through a partnership of faith-based 
leaders and the business community of the Lower Rio Grande Valley, Texas to address employers’ needs for skilled 
workers and link the area’s un- and underemployed with high skilled, high wage jobs in the region. VIDA’s primary goal is 
for participants to graduate with an associate’s degree or industry-recognized certificate in a high-demand occupation and 
find employment that pays a living wage in their area of study. VIDA supports full-time enrollment at local colleges through 
required attendance at weekly intensive counseling sessions along with direct assistance with tuition and related training 
expenses of $13,750 for two and a half years on average. To start, program participants complete an initial assessment 

to determine their ability to commit to the program. For college-ready participants, experienced VIDA counselors conduct 
weekly mandatory group or individual case management and counseling sessions at students’ colleges to provide 
information on how to succeed in college and employment, identify problems early, and provide social support. Program 
participants can receive wraparound support services for tuition, books, and other needs such as transportation to reduce 
the cost of obtaining a certificate or degree. For program participants who are not college-ready, VIDA offers a College Prep 
Academy, which is a 16-week, accelerated, full-time basic skills bridge program. During the 16 weeks, participants attend a 
daily course to prepare to pass college entrance exams. For College Prep Academy participants, VIDA pays for two rounds 
of college entrance exam testing. 


Participants in the comparison group could not access the VIDA services but could access other services available in 

the community, such as those provided by V/DA’s college partners and other community resources such as occupational 
training programs, developmental education, and academic advising through area community colleges; career and personal 
counseling through area community colleges and local social service providers including American Job Centers and Texas 
Department of Assistive and Rehabilitative Services; and financial aid to attend school through federal and local loans, 
grants, and scholarships. 


Study authors reported findings on five outcome measures that are eligible for review under the Postsecondary Career and 
Technical Education topic area. One eligible outcome, total college credits earned within 24 months of random assignment, 
was in the credit accumulation domain. One eligible outcome, receipt of a college credential: any degree within 24 months 
of random assignment, was in the postsecondary degree attainment domain. Three eligible outcomes, receipt of a college 
credential: any credential within 24 months of random assignment; receipt of a college credential: any certificate within 24 
months of random assignment; and receipt of a credential from a college within 24 months of random assignment, another 
education or training institution by time of survey, or a licensing/certification body by time of survey, were in the industry- 
recognized credential, certificate, or license completion domain. 


The study also reported findings for total college credits earned by the end of successive periods after random assignment 
(by month 6, month 12, and month 18); receipt of a college credential: associate’s degree 24 months after random 
assignment; receipt of a college credential: bachelor’s degree 24 months after random assignment; receipt of a college 
credential: level 1 certificate 24 months after random assignment; and receipt of a college credential: level 2 certificate 24 
months after random assignment. Summaries of these findings are available on the WWC website 
(httos://whatworks.ed.gov). The supplemental findings do not factor into the intervention’s rating of effectiveness. 


Study authors reported findings on 10 outcome measures that do not meet standards, and thus are not included in the 
intervention report. The findings include working in a job paying $12/hour or more within 24 months of random assignment; 
working in a job requiring at least mid-level skills within 24 months of random assignment; receipt of a college credential: 
associate’s degree 36 months after random assignment; receipt of a college credential: bachelor’s degree 36 months after 
random assignment; credential earned-a licensing/certification body 24 months after random assignment; receipt of a 
credential from another education or training institution by time of survey; receipt of college credential: level 1 certificate 36 
months after random assignment; receipt of a college credential: level 2 certificate 36 months after random assignment; 
receipt of a college credential: any certificate 36 months after random assignment; and receipt of a college credential: any 
credential 36 months after random assignment. These findings do not meet WWC group design standards because the 
analysis does not provide a credible measure of the effectiveness of the intervention. 


VIDA received funding from the Pathways for Advancing Careers and Education (PACE) evaluation for enhanced 
recruitment in order to double program enrollment and establish a comparison group to conduct an evaluation of the 
intervention. 
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Research details for Juniper et al. (2020) Findings from Juniper et al. (2020) show evidence of an 


Juniper, C., Rodriguez, P., & Prince, H. (2020). Evaluation of 
Travis County investments in workforce development: 2020 
update. https://raymarshallcenter.org/files/2021/01/FINAL 


indeterminate effect on short-term employment (Table 
9). These findings are based on an outcome analysis that 
includes 1,212 students. 


FY-2019 Travis-County-Report.pdf 


Table 9. Summary of findings from Juniper et al. (2020) 


Meets WWC Group Design Standards With Reservations 


Study findings 
Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
Short-Term Employment 1,212 students 0.14 +6 No 


Table 10. Description of study characteristics for Juniper et al. (2020) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Meets WWC Group Design Standards With Reservations. This is a study that uses a quasi-experimental design (QED) 
in which the analytic intervention and comparison groups satisfy the baseline equivalence requirement. 


This is a study of Capital IDEA, which is a sectoral workforce development program implemented in Travis County, Texas, 
and is based on the Project QUEST model. Participants attend training at LifeWorks workforce development sites, including 
Austin Community College’s (ACC) Highland Campus (95% of students) and Temple College (5% of students). 


This study is a QED in which 606 program participants were matched with a comparison group of 606 individuals using 
propensity score matching. Potential comparison group members received services supported by Travis County. The study 
uses administrative data from the Unemployment Insurance program beginning with four quarters prior to the intervention and 
following up to nine quarters after intervention participation. The study focuses on the cohort of participants completing the 
program in 2016-2019. 


Eligible applicants had to be at least 18 years old, have a High School Equivalency Certification (HSEC) or high school 
diploma, lack an associate's or higher degree, meet reading and math skills requirements, and report a household income 
at or below 200% of the federal poverty threshold. Of the Capital IDEA participants in the study, 56% were White, 24% 
were Black, and 5% were Asian. Forty-one percent of the sample were Hispanic, and 70% were female. Nearly half (49%) 
had attended some college, and over one-quarter (27%) were receiving public benefits. The program recruited low-wage 
workers and encouraged them to continue working part-time throughout the program. No information was provided on the 
demographic characteristics of the comparison group. 


Capital IDEA is a sectoral workforce development program that offers non-traditional students with the opportunity to 
pursue training in high-wage, high-demand occupations in health care, information technology, and other leading industries 
such as professional trades or applied technologies. The program coordinates and collaborates with Austin-area community 
colleges and employers to help prepare participants for jobs with good wages and benefits. Applicants who are unlikely to 
pass a college admissions assessment can enroll in Capital IDEA’s College Prep Academy, an intensive 6.5 hour per day, 
five-day a week, 12-week program designed to build math, reading, writing, and study skills. Most participants required 
only one semester of the College Prep Academy (10% of participants received an additional semester of support). The 
program assigns a career navigator to each student. Career navigators meet students at the beginning of each semester, 
then communicate and work with students throughout the semester. Group support sessions are also offered, with topics 
for these sessions driven by student needs and their ability to navigate the college experience. The career navigators 

also communicate over telephone, text message, email, and video conferencing to help guide students through the higher 
education system. The program encourages students to work part-time throughout the program. The program covers 
tuition, fees, books, uniforms, tools, training software, and other course-specific required items. In addition, Capital IDEA 
provides support with basic office supplies, backpacks, and eye examinations and eyeglasses, and offers emergency 
financial assistance for students who need help with utility bills, rent, and mortgage payments, and childcare. There is also 
free Cap Metro public transportation for students enrolled at ACC. 
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Comparison 
group 


Outcomes and 
measurement 


Additional 
implementation 
details 


Comparison group members received employment assistance services available to the community through the state’s 
WorkInTexas (WIT) program and local Workforce Investment Act-funded Workforce Solutions Career Centers. Data for the 
comparison group were drawn from The Workforce Information Systems of Texas (TWIST) records. 


Study authors reported findings on one outcome measure that is eligible for review under the Postsecondary Career and 
Technical Education topic area: employment one year after program completion, which is in the short-term employment 
domain. 


Study authors reported findings on three outcome measures that do not meet standards, and thus are not included in 

the intervention report. The findings for employment (two years after program completion) in the short-term employment 
domain, average quarterly earnings (one year after program completion), and average quarterly earnings (two years after 
program completion) in the short-term earnings domain do not meet WWC group design standards because baseline 
equivalence of the analytic intervention and comparison groups is necessary, but the requirement was not satisfied. 


No additional implementation details are provided in the study. 
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Endnotes 


' Carnevale, A. P., Cheah, B., Ridley, N., & Strohl, J. (2017). 
Good jobs that pay without a BA. Washington, DC: 
Georgetown University, Center on Education and the 
Workforce. 

? Shapiro, D., Ryu, M., Huie, F., Liu, Q., and Zheng, Y. (2019). 
Completing College 2019 National Report (Signature Report 
18), Herndon, VA: National Student Clearinghouse Research 
Center. 


3 This report includes studies on Project QUEST, VIDA, 
and Capital IDEA because the research that meets WWC 
standards involves these three variants of the intervention; 
the details differ slightly between the three programs, and 
these differences are described in this report. We describe 
all programs as Project QUEST for simplicity. 


* Absence of conflict of interest: This intervention report 
includes a study conducted by staff from Abt Associates 
(Rolston et al., 2017). Because Abt Associates is a contractor 
that administers the WWC, this study was reviewed by staff 
members from a different organization. 


5 The literature search reflects documents publicly available 
by April 2021. Reviews of the studies in this report used 
the standards from the WWC Procedures and Standards 
Handbook (version 4.0) and the Postsecondary Career and 
Technical Education review protocol (version 4.0). The 
evidence presented in this report is based on available 
research. Findings and conclusions could change as 
new research becomes available. A longer-term impacts 
report for the Rolston et al. (2017) study is scheduled to be 
published in Summer 2021, too late to be included in this 
intervention report. 


6 Full-time enrollment is prescribed by the occupational 
training program. In a community college setting, full- 
time enrollment can be equivalent to 12+ credit hours, 
unless fewer credit hours are required per semester for the 
training program. 


7 The effects of Project QUEST are not known for other 
outcome domains within the Postsecondary Career and 
Technical Education topic area, including technical skill 
proficiency and long-term employment. 
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